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Abstract 

We briefly present Java PathExplorer (JPaX), a tool developed at NASA Ames for monitoring the execution 
of Java programs. JPaX can be used not only during program testing to reveal subtle errors, but also can be 
applied during operation to survey safety critical systems. The tool facilitates automated instrumentation of 
a orogram in order to properly observe its execution. The instrumentation can be either at the bytecode level 
or at the source level when the source code is available. JPaX is an instance of a more general project, ca led 
PathExplorer (PaX ), which is a basis for experiments rather than a fixed system, capable of monitoring various 
programming languages and experimenting with other logics and analysis techniques. 


1 Introduction 

The success of most technological experiments today, including space craft and rover technology, heavily depends 
on the correctness of software. Two common ways to approach the delicate problem of software correctness 
are testing and formal methods. Although traditional testing techniques scale well and are by far the most 
used techniques in practice to validate software systems, they are usually ad hoc and do not allow for forma 
specification and verification of high level logical properties that a system needs to satisfy. On the ot er an , 
traditional formal methods such as model checking and theorem proving are usually very heavy and rarely can 
be used in practice successfully without considerable manual effort. 

The Automated Software Enjpneering group at NASA Ames Research Center has for some time investigated 
advanced formal methods for insuring software correctness, in both program synthesis [14, 5, 18] and program 
verification [8, 9, 17, 7, 11]. This paper, together with [10, 15, 11, 12] describes our effort in runtime verification 
which can be roughly defined a i combining testing and formal methods. The merge of testing and temporal 
logic specification is an attempt to achieve the benefits of both approaches, while avoiding some of the pitfalls of 
ad hoc testing and the complexity of theorem proving and model checking. In this paper, we present the current 
status of a new runtime verification system, called Java PathExplorer (JPaX), for monitoring Java programs 
by analyzing (exploring) particular execution traces. We are considering two apparently distinct approaches to 
monitoring which can be well combined. In black-box monitoring, the general idea consists of extracting state 
events from an executing program, which has been automatically instrumented, and then analyzing them via 
an observer process. This approach is suitable in general when certain fields (variables, data structures, etc.) 
need to be observed at any morrent over the execution of the program. In white-box monitoring, one assumes 
that the source code of the program is available and that annotations can be introduced manually in the code at 
appropriate places, where they will then be expanded and compiled together with the program. This approach 
is more appropriate when certain fields need to be observed at specific places in the program. Many of the 
algorithms used are the same in both black-box and white-box approaches. JPaX currently performs two kinds 
of verification, namely logic based monitoring and error pattern analysis. 

Logic based monitoring consis ts of runtime checking formal requirement specifications written in high leve 
logics by users of the system. Logics are currently implemented in Maude [1], a high-performance system 
supporting both rewriting logic and membership equational logic. One can naturally and easily define new 
logics in Maude, such as for example temporal logics, together with their finite trace operational semantics^ 
Currently, JPaX supports two juiltin logics, future time and past time linear temporal logic. Logic based 


mtmif.oiTntj <:;ui l>(‘ instant, iat.i'd at nit, her the sonrm code level in the e.uxe of whit.c-hox monitoring, or ;if. the 
byte code level in the e;ise of black-box monitoring. The idea of using temporal logic in program testing is not 
new. We only mention the commercial Temporal Rover tool (TR.) [3] which implements temporal logics in a 
white-box fashion, and the MaC tool [13] which implements a special past time temporal logic in a black-box 
style. 

Error pattern analysis consists of analyzing one execution trace using various error detection algorithms 
that can identify error-prone programming practices, such as unhealthy locking disciplines that may lead to 
data races and/or deadlocks. The important and appealing aspect of these algorithms is that they find error 
potentials even in the case where errors do not explicitly occur in the examined execution trace, but it is worth 
mentioning that these algorithms are usually neither sound nor complete for the class of errors they are designed 
for. They are generally fast and scalable, and often catch the problems they are designed to catch, that is, the 
randomness in the choice of run does not seem to imply a similar randomness in the analysis results. Two such 
known algorithms focusing on concurrency errors have been implemented in JPaX, one for deadlocks and the 
other for data races, but the system is designed in such a way that users can relatively easily attach new such 
algorithms. 

The paper is organized as follows. Section 2 gives an overview of JPaX. Section 3 describes the underlying 
logic formalisms for writing requirement specifications, while Section 4 describes some of the error detection 
algorithms for debugging concurrent programs. Finally, Section 5 contains conclusions and a description of 
future work. 


2 Overview of JPaX 

We next shortly describe JPaX’s architecture. Unless otherwise specified, the black-box monitoring approach 
is considered. We use the acronym JPaXx (Java PathExplorer Expander) when we refer to the white-box 
monitoring aspect of JPaX. White-box monitoring can be used in conjunction with black-box monitoring and 
works as follows: after one annotates the source code with desired formal requirements, one passes that source 
file through the white-box monitoring filter, JPaXx, and the annotations will be then expanded automatically 
into Java source code using optimal algorithms mentioned later in the paper, where after the expanded code 
may be compiled as normal and executed; exceptions will be thrown when formulae are violated. 

We next discuss the architecture of the black-box monitoring part of our system. JPaX can be regarded as 
consisting of three main modules: an instrumentation module, an observer module, and an interconnection mod- 
ule that ties the other two together through the observed event stream. The instrumentation module performs 
a script-driven automated instrumentation of the program to be observed. The instrumented program, when 
run, will emit relevant events to the inter-connection module, which further transmits them to the observation 
module. The observer may run on a different computer, in which case the events are transmitted over a socket. 
Hence, the input to JPaX consists of references to two entities: the Java program in byte code format to be 
monitored (created using a standard Java compiler) and the specification script defining the kinds of verification 
requested. The output is a (possibly empty) set of warnings printed on a special screen. 

More specifically, the specification script defines what (if any) kind of error pattern detection algorithms 
should be activated, and what (if any) kind of logic based monitoring should be performed, and in that case 
what the requirements are. For logic based monitoring, we have been inspired by the MaC language framework 
[13] and have split the specification into an instrumentation script and a verification script. The verification 
script identifies the high level requirement specifications that events are to be checked against. The propositions 
referred to in these specifications are abstract boolean flags, and do hence not refer directly to entities in the 
concrete program. The instrumentation script establishes this connection between the concrete boolean program 
predicates and the abstract propositions. The advantage of this layered approach, as also stated in [13], is that 
the requirement specification can be created without considering low level issues, and can even be created before 
the construction of the program. Currently, the scripts are written in Java. Thus, high level Java language 
constructs can be used to define the boolean predicates to be observed. 

The Java byte code instrumentation is performed using the powerful Jtrek Java byte code engineering tool 
[2] from Compaq. Jtrek makes it possible to easily read Java class files (byte code files), and traverse them as 
abstract syntax trees while examining their contents, and insert new code. The inserted code can access the 
contents of various runtime data structures, such as for example the call-time stack, and will, when eventually 
executed, emit events carrying this extracted information to the observer. 


Thr observer receives the 'vents and dispatches these to a sot of observer rules, each Milo pot foi ming a 
particular analysis that, has boon requested in the verification script. Generally, this modular rule bused design 
allows a user to easily define new runtime verification procedures without interfering with legacy code. Observer 
rules are written in Java, but can call programs written in other languages, such as for example Maude. Maude 
plays a special role in that hig h level requirement specifications can be written in the Maude rewriting logic. 
The Maude rewriting engine can then be used in two different ways: as a monitoring engine during program 
execution, or as a translation engine before execution. In the former case, execution events are submitted to 
the Maude program, which in turn evaluates them against the requirement specification. In the latter case, 
the specification is translated into a data structure optimal for program monitoring, which is then sent back to 
Java, and used within the Java program to check the events during execution. 

JPaX is built on a generic environment, named PathExplorer (PaX), which only consists of the inter- 
connection module and the observer module. The goal is to make it possible to monitor programs in other 
programming languages, such as for example C and C++, by just providing a language specific instrumentation 
module. Such an experiment has been performed in collaboration with Rich Washington, a member of the 
Robotics group at NASA Ames, on a 90,000 line C++ application for controlling a rover. The experiment just 
activated the deadlock detection rule, and located a deadlock potential in the application that had not been 
discovered through testing. 


3 Logic Based Monitoring 

Logic based monitoring consists of checking execution events against a user-provided requirement specification 
written in some logic, typically e.n assertion logic with states as models, or a temporal logic with traces as models. 
JPaX allows the user to define* such new logics in a flexible manner using the Maude algebraic specification 
language. Maude [1] is a modularized specification and verification system that very efficiently implements 
rewriting logic. JPaX currently provides linear temporal logics, both future time and past time, as builtin 
logics. Notice that multiple logics can be used in parallel, so each property can be expressed in its most suitable 
language. 

3.1 Future Time LTL Semantics 

We first present a semantics of finite trace LTL using standard mathematical notation. Assume two total 
functions on traces, head : Traci* -4 Event returning the head event of a trace and length returning the length 
of a finite trace, and a partial function tail : Trace -4 Trace for taking the tail of a trace. That is, head(e : t) = 
head(e) = e, tail(e,t) = t, and tength(e) - 1 and length(e,t) = 1 + length(t). Assume further for any trace t , 
that ti denotes the suffix trace that starts at position i, with positions starting at 1. The satisfaction relation 
f= C Trace x Formula defines when a trace t satisfies a formula /, written t f= /, and is defined inductively over 
the structure of the formulae as :bllows, where A is any atomic proposition and X and Y are any formulae, and op 
are standard binary propositional connectives(V, and, -4) and [] , <>, O, U stand for always in the future, 
eventually in the future, next , and until respectively: 


t h-i 

iff 

A € head(t) 

t ' rue 

iff 

true, 

t |= : al3« 

iff 

false, 

+ -U 

iff 

not t ^ X, 

t j= 3 op r 

iff 

t X op t |= Y, 

t f= 1 ]x 

iff 

(V i < length(t)) ti X 

t \= < >x 

iff 

(3 i < length(t)) 2j |= X , 

+ ] n 

iff 

(3 i < length(t)) (ti t= Y and (V j < i) t, j= X) 

t J= Oi 

iff 

(if tarf(t) is defined then ta\l(t ) f= X else t (= X) 


Notice that iiveness properties do not really make sense in finite trace LTL without statistical analysis, as 
already noticed in [12, 4]. In particular, the formula □ oa is violated if and only if a is false in the last observed 
state of the monitored program. The formula o(Da V D^a) is always true in finite trace LTL and our optimal 
generator mentioned in subsection 3.3 proved that. 



3.2 Past Time LTL Semantics 

Hero wo present the semantics of finite trace past time temporal logic. For past time the definitions change 
as follows: last(t,e) = last(e) = e t }ront(t,e) — t } and lcngth(e) = I and length(t,e) = 14- length(t). Assume 
further for any trace t, that t t denotes the prefix trace that ends at position i. The satisfaction relation 
f= C Trace x Formula defines when a trace t satisfies a formula /, written t J= /, and is defined inductively over 
the structure of the formulae as follows, where A is any atomic proposition and X and Y are any formulae, op 
are standard binary propositional connectives(V. and, -»), and [*], <*>, (*), S stand for always in the past, 
eventually in the past, previous , and since respectively: 


t j= A 
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£rue, 

t |= false 
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false , 
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not t X, 

t f= X op Y 

iff 
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iff 

(if front(t) is defined then front(t) {= X else t X) 


3.3 Efficient Observer Generation 

After experimenting with runtime verification algorithms for LTL [10, 15, 11], each with its advantages and 
drawbacks, we realized that in order for one to properly compare these, one needs to first understand and 
establish criteria for “good” runtime verification algorithms. Consider a fixed logic. The following is are some 
priorities that we also discussed in [12] which currently influence the choice of runtime algorithms in JPaX : 
1) forwards design - algorithms that visit the execution traces backwards involve storing the trace and cannot 
throw exceptions or. guide the program when a formula is violated, so we exclude them; 2) runtime efficiency - 
the normal execution of the program should be influenced as little as possible, so we exclude algorithms that are 
worse than linear in the size of the trace and prefer algorithms that are not exponential in the size of formula 
if possible; 3) initialization - the time required to generate code or data structures from formulae cannot be 
ignored, but it is considered less important than the previous criteria. 

Next we very briefly present some concepts that lead to a future time finite-trace LTL formula-checking 
algorithm that is the best one of which we are aware satisfying the criteria above. It visits the execution trace 
forwards and its worst-case runtime complexity is 0{nk ), where n is the length of the trace and k is the number 
of variables of the formula. The complete details together with optimality proofs will appear elsewhere soon. 
The interested reader can consult [12] for more detail. 

We have designed and implemented in Maude (in less than 200 lines of code) a relatively easy and elegant 
procedure that generates an optimal BTT finite trace FSM from any LTL formula. Despite its worst-case 
exponential complexity, it is quite fast on typical formulae: it needed more than 1 second only on hand-crafted 
artificial formulae. This initialization time is spent only once, at the beginning of the monitoring. The following 
are a few examples of optimal BTT finite trace finite state machines generated by our current implementation: 


Formula 

State 

next 

end 

□ o a 

1 

1 

a?true : false 

o(Qa V D-ia) 

1 

1 

true 

□ (a — ► ob) 

1 

a?(6?l : 2) : 1 

a?(6?true .- false) : true 


2 

bn : 2 

b? true : false 

a U (b U c) 

1 

c?true : (a?l : (b? 2 : false)) 

c?true : false 


2 

c?true : (b?2 : false) 

c?true : false 


Notice that the generated finite state machines are deterministic. The drawback of generating deterministic 
machines is that they may be quite large for some formulae, even large enough that one cannot afford to generate 
and store them in practical situations. For that reason, we intend to also attach a module that generates 
nondeterministic automata from LTL formulae, like in [4], to JPaX to be activated when the deterministic one 
cannot be generated in a reasonable amount of time. 



3.4 White-box Monitoring 

The procedure above can be used in instrumented bytecode monitoring (black-box) as well as in source code 
level monitoring (white-box). I' the source code is available, the second approach may be more desired for two 
reasons: exception handling and efficiency. With regards to exceptions, white-box logic monitoring, can now be 
seen as a natural extension to I he language that allows one to describe conditions that should raise exception, 
as already noticed in [3]. 

As stated earlier, the worst-case runtime complexity for future time LTL monitoring is 0(nk ), where n is the 
length of the trace and k is the number of variables of the formula. For the past time monitoring, the worst-case 
runtime complexity becomes O nm), where n is the same as above and m is the number of subformulae of the 
formula. We conjecture that th 3 complexity of past time monitoring can be further improved. 

To demonstrate the tool we use a very simple example involving the monitoring of a traffic light controller. 
The following is an annotation left) and the expanded form of that annotation (right). 


<code> 


<code> 


/* JPaX: after green comes yellow 

Atom isRed * Cboolean exp>; 

Atom isGreen * Cboolean exp> ; 

Atom isYellov = Cboolean exp> ; 

Formula : 

[]( isGreen -> ( ! isRed U isYellov )); 


try { 

svitch(bttFsmState) { 
case 1 : 

bttFsmState = isGreen ? isYellov ? 1 : isRed ? 0 : 2 : 1 ; break; 
case 2 : 

bttFsmState = isYellov ? 1 : isRed ? 0 : 2; break.; 

> 

if (bttFsmState « 0) throv nev ExceptionO'Prop . Failure"); 

> catch(Exception e) { System. out .println(e .getMessageO ) ; } 


Ccode> 


<code> 


The annotation is commentei out in a normal Java comment but contains the keyword JPal followed by a 
title, a list of atomic declarators, and an LTL formula. Here the formula states “it is always the case that if 
the light is green it will not be red again until it has first been yellow”. Now this new code may be compiled 
normally and will raise an exception if the property is violated. 

Next we look at the same example, but now we express the property in past time finite trace LTL. Here 
we took the dynamic programming algorithm for future time LTL presented in [15] and converted it to past 
time LTL. We are planning a follow up paper that will discuss these algorithms in more detail. Once again 
the annotation is on the left and the expanded annotation is on the right. Not seen in this example is the 
initialization of the array prev, which holds the evaluation of a particular subformula in the previous state. 
Here the formula states “if the light is red then the light has not been green since it was yellow” . 


Ccode> 


Ccode> 


/* JPaX : after green comes yellow 

Atom isRed = Cboolean exp>; 

Atom isGreen = cboolean exp> ; 

Atom isYellov = Cboolean exp>; 

Formula : 

isRed -> ( ! isGreen S isYellov) ; 

*/ 

Ccode> 


nov[6] = isGreen; 
nov[5] = isYellov; 
nov[4] * ! nov[6] ; 

nov[3] = nov[5] II (now [4] fefc prev[3]); 

nov[2] * isRed; 

nov[l] = ! nov[2] II nov[3] ; 

prev [3] * nov[3] ; 

if (! prev[l]) { throv nev Except ion("Prop . Failure");} 
Ccode> 


The ordering of the above statements is the result of parsing the formula and then visiting each subformula 
in the parse tree in a breadth first ordering from the bottom up. When the expanded code is executed, each 
statement is evaluated to determine whether the subformula holds in that state or not. 



4 Error Pattern Analysis 

Error pattern runtime analysis algorithms explore an execution trace and detect error potentials. The important 
and appealing aspect of these algorithms is that they find error potentials even in the case where errors do not 
explicitly occur in the examined execution trace. They are usually fast and scalable, and often catch the 
problems they are designed to catch, that is, the randomness in the choice of run does not seem to imply a 
similar randomness in the analysis results. The trade off is that they have less coverage than heavyweight 
formal methods and often suggest problems which, after a careful semantical analysis, turn out not to be errors. 
Two examples of such algorithms focusing on concurrency errors have been implemented in JPaX: the Eraser 
[16] data race analysis algorithm and a deadlock analysis algorithm based on analyzing lock cycles. Both these 
algorithms have been previously implemented by Compaq in the Visual Threads tool [6] to work for C and 
C++. 

4.1 Data Race Analysis 

We briefly describe here how easily data races can occur in concurrent programming and how Eraser [16] has 
been implemented in JPaX to work on Java programs. A data race occurs when two or more concurrent threads 
access a shared variable, at least one access is a write , and the threads use no explicit mechanism to prevent the 
accesses from being simultaneous. The Eraser algorithm detects data races by studying a single execution trace 
of the monitored program, trying to conclude whether there exist valid runs where data races are possible. We 
illustrate the data race analysis wfith the following example. 

1. class Value! 

2. private int x = 1; 

3. 

4. public synchronized void add(Value v){x * x + v.getO;} 

5. 

6. public int get (Hreturn x;> 

7 . > 

8 . 

9. class Task extends Thread! 

10. Value vl; Value v2; 

11 . 

12. public Task(Value vl, Value v2)! 

13. this.vi * vl ; this.v2 = v2; 

14. this . start () ; 

15 . } 

16. 

17. public void run()!vl . add(v2) ; } 

18. > 

19. 

20. class Main! 

21. public static void main(String[] args)! 

22. Value vl = new Value (); Value v2 * nev Value(); 

23. new Task(vl,v2); new Task(v2,vl); 

24. > 

25. > 

The class Value contains an integer variable x, a synchronized method add that updates x by adding the content 
of another Value variable, and an unsynchronized method get that simply returns the value of x. Task is a 
thread class: its instances are started with the method start which executes the user defined method run. Two 
such tasks are started in Main, on two instances of the Value class, vl and v2. W T hen running JPaX with the 
Eraser option switched on, a data race potential is found, reporting that the variable x in class Value is accessed 
unprotected by the two threads in lines 4 and 6. respectively. The generated warning message gives a scenario 
under which a data race might appear, summarizing the following. One Task thread can call the add method 
on the object vl with the parameter Value object v2, whose content is thus read via the unsynchronized get 
method. The other thread can simultaneously do the dual thing, i.e., call the add method on v2. Therefore, the 
content of v2 might be accessed simultaneously by the two threads. 

Roughly, the algorithm works and is implemented in JPaX as follows. The instrumented byte code of the 
monitored program emits to the observer appropriate events when variables are read or updated, and when locks 
are acquired or released as a result of executing Java’s synchronized statements or from calling/returning from 
synchronized methods. The observer maintains two data structures: a thread map that keeps track of all the 


locks owned bv each thread, aid a variable map that associates with each (shared) variable the intersection of 
the set of locks that has been :ornmonly owned by all accessing threads in the past. If this set ever becomes 
empty then a data race potential exists. More precisely, when a variable is accessed for the first time, the locks 
owned by the accessing thread it that moment are stored in the variable's variable set. Subsequent accesses by 
other threads causes the set to be refined to its intersection with the locks owned by those threads. An extra 
state machine is also maintained for each variable to keep track of how many threads have accessed the variable 
and how (read/write). This is used to reduce the number of false warnings, such as situations in which variables 
are initialized by a single thread without locks (which is safe) or several threads only read a variable after it 
has been initialized (which is also safe). 

Deadlock Detection 

Deadlock potentials are hard to find in general, but there are classical deadlock situations which occur when 
multiple threads take locks in d fferent order. For example, a deadlock will arise if a thread acquires a lock and 
then, without releasing it, acquires another lock, while another thread first acquires the second lock and then 
the first one. One can simply create such a situation in the previous Java example if one wrongly tries to repair 
the data race by also defining the get method in line 6 as synchronized: 

6. public synchronized int g« t () {return x;} 

It is clear now that the data ra :e algorithm will indeed not return a warning anymore because the variable x 
can no longer be accessed simultaneously from two threads. However, there is a deadlock potential now and 
JPaX detects it. More exactly, when running JPaX on the modified program, a lock order problem is found 
and an appropriate warning message is issued summarizing the fact that two object instances of the Value class 
are taken in a different order by the two Task threads. It also indicates the line numbers where the threads 
may potentially deadlock: line 1 where the get method called from add may lock the second object. Notice 
that this deadlock doesn’t need :o appear in the examined trace in order for this warning to be issued. In fact, 
deadlock potentials might be reported in general even if those deadlocks will never appear in any execution of 
the program. Any execution of the modified program above will cause a warning to be issued. 

The runtime deadlock analysis algorithm is also implemented in the observer and it needs only a subset of 
the events generated for the data race algorithm, namely those related to acquires and releases of locks that 
result from executing Java’s syrehronized statements or from calling/returning from synchronized methods. 
Two data structures are maintained in the observer: as in the data race algorithm a thread map keeps track of 
the locks owned by each thread, while a second data structure, a lock graph, updates a graph that accumulates as 
nodes all the locks taken by any thread during an execution, the edges recording locking orders. In other words, 
an edge is introduced from a lock to another each time when a thread that already owns the first lock acquires 
the other. If during the execution of the program this graph becomes cyclic, then there is a deadlock potential 
related to lock ordering in the program. This simple algorithm can reveal more complex deadlock potentials 
between more than two threads, as illustrated for example by the classical dining philosopher’s example. 

5 Conclusions 

We have presented JPaX, a rurtime verification tool under development at NASA Ames Research Center. 
JPaX provides an integrated environment for instrumenting Java byte code to emit events during execution to 
an observer, which performs two kinds of analysis: logic based monitoring, checking events against high level 
requirements specifications, and error pattern analysis, searching for low level programming errors. It has been 
shown how the two kinds of verification can be combined by viewing them as rules within an extensible set of 
rules. A white-box logic based monitoring approach has also been presented. In the case where efficiency is 
required, we have shown that optimal data structures can be generated from future time and past time LTL. 
Finally, two known error pattern detection algorithms, one for data races and one for deadlocks, have been 
implemented to work on Java. 

Of other future work can be mentioned that we will experiment with new logics in Maude more appropriate 
to monitoring than LTL, such as interval and real time logics and UML notations. Future work on error pattern 
analysis will try to develop new algorithms for detecting concurrency errors other than data races and deadlocks, 
and of course to try to improve existing algorithms. We will also study completely new functionalities of the 
system, such as guided execution via code instrumentation to explore more of the possible interleavings of a 


nori-dotorinmistir concurrent program during testing. Dynamic program visuaii/aruou ; _s also •’ Future? subject, 
where wo regard a visualization package just another rule in the observer. A more user friendly interface, 
both graphical and functional, will be provided, and finally the tool will be evaluated against NASA safety 
critical applications. 
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